The present invention relates to a master-slave manipulator system.
Many master-slave manipulator systems have been developed, in which an operator manipulates manipulation means and a slave manipulator is operated in accordance with the manipulation. When a slave manipulator is operated by manipulation means while an object and the slave manipulator are seen with a camera, which comprises observation means, it is sometimes desirable to operate not only the slave manipulator but also the camera itself.
In the case where a slave manipulator being operated is one in number, there is a method, in which manipulation means associated with the slave manipulator is manipulated by the right hand and manipulation means associated with a camera is moved by the left hand. Also, a system is developed, in which in the case where two slave manipulators are operated by two associated manipulation means with the use of the both hands, the hands are separated from the manipulation means and the manipulation means associated with a camera is manipulated.
However, there are some cases where a speed of manipulation is demanded and it is not desirable to separate a hand or hands from manipulation means, which is being manipulated. In this case, means for moving a camera without using a hand or hands in manipulation is contrived as in JP-A-7-328016, JP-A-7-328971, etc. In JP-A-7-328016, while a picture is presented by a head mounted display, a position of the head mounted display is detected, and a camera visual field is changed according to movements of the head. JP-A-7-328971 discloses a manipulator with a TV camera, in which the camera is mounted to the manipulator whereby it is possible to automatically move the manipulator to a position suited to a work to smoothly carry out a work.
Also, means for associating movements of a working tool in an image photographed by a camera with the relationship of a manipulation input device is contrived in JP-A-7-328016 and Japanese Patent No. 3766805. JP-A-7-328016 discloses support of manipulation performed by superimposing an operation coordinate system of a manipulator on a screen, on which a camera image is displayed. Japanese Patent No. 3766805 discloses to conform a direction of motion of a manipulator displayed on an image to direction of motion of a manipulation input device.
As described above, since a master-slave manipulator system using a camera involves a limit in camera visual field, it is frequently desirable to change a camera visual field depending upon movements of a work object and a change in situation.
When a camera is to be changed in position, however, the related art involves a problem that a slave manipulator working portion goes out of a screen, and an agreement in position and orientation between the slave manipulator working portion projected onto display means and manipulation means is not taken account of after a visual field is changed, so that a master-slave operation is hard to perform after the visual field is changed.
Also, when a pointer on GUI (Graphical User Interface) is to be moved by a mouse, it is desired that manipulation means can perform manipulation (referred below to as clutch manipulation) of lifting the mouse in the air when a limit of a manipulating range of the mouse is reached, moving the mouse to a position of easy manipulation, and then again moving the pointer on GUI. That is, there is caused a problem in a limit of a moving range of manipulation means and it is desirable to perform a clutch manipulation, in which manipulation means is returned to a position for easy movement, while a position of a slave manipulator is left as it is. In the case where such clutch manipulation is performed, there is caused a problem that the slave manipulator projected onto display means and manipulation means disagree in position and orientation with each other whereby a direction, in which manipulation is performed by the manipulation means, and a direction, in which the manipulator is moved, disagree with each other. Such disagreement in direction of movement increases a load on an operator to make it difficult to perform a complex operation.
Further, there is also caused a problem that while a visual field is changed, a slave manipulator, which carries out a work, is suspended to lead to worsening in working efficiency and extending in working hour. In particular, in a medical field in recent years, there is spread a low invasion surgery, in which in order to reduce a load on a patient's body, a small hole is formed on a patient's body, a surgical tool and an endoscope are inserted therefrom to perform medical treatment while observing states of an affected part and the surgical tool in endoscope image. In a surgical operation with the use of a slave manipulator, a tool of the slave manipulator is likewise inserted from a small incised part to perform medical treatment on the basis of an endoscope image. In such situation, a manipulation method imposing a load on an operator is not preferred because of a limit in degree of freedom and so it is desirable to readily realize a motion of a slave manipulator, which reflects an operator's will exactly.
In JP-A-7-328016 described above, a camera visual field is changed without the use of both hands but no consideration is taken with respect to disagreement in position and orientation between the hand, which grips a master, after a visual field is changed and a slave manipulator working portion seen in a screen.
In JP-A-7-328971 described above, it is thought that a camera always photographs a slave manipulator but no consideration is taken with respect to the relationship between a slave manipulator displayed on display means and manipulation means.
Japanese Patent No. 3766805 described above discloses an agreement in direction of motion between a surgical manipulator and remote manipulation means but no mention is made of means for moving an endoscope picture and an agreement between a direction of movement of a manipulator on a displayed picture at that time and a method of manipulating manipulation means. Also, no consideration is taken with respect to clutch manipulation.
That is, such related art involves a problem that when a camera visual field is changed and clutch manipulation is begun, a difference in orientation between a slave manipulator working portion photographed by a camera and manipulation means is generated to make it hard to perform manipulation.